Papers with adaptive inference
The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for difficulty estimation rely on repeated response sampling, auxiliary models, or fine-tuning the target model itself. |
| Approach: | They propose a method that leverages only the hidden representations produced by large language models. |
| Outcome: | The proposed method outperforms baselines in difficulty estimation on textual and multimodal tasks and improves adaptive reasoning strategies with fewer generated tokens. |
Accelerating Inference for Pretrained Language Models by Unified Multi-Perspective Early Exiting (2022.coling-1)
Copied to clipboard
| Challenge: | Existing competitive methods to accelerate inference of pretrained language models are limited by their complexity and computational consumption. |
| Approach: | They propose a unified horizontal and vertical multi-perspective early exiting framework to accelerate inference of transformer-based models. |
| Outcome: | Experiments show that MPEE can achieve higher acceleration inference with competent performance than existing competitive methods. |
SHARCS: Efficient Transformers Through Routing with Dynamic Width Sub-networks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Several methods have been proposed to improve the inference efficiency of transformer-based models. |
| Approach: | They propose a new adaptive inference method that takes into account the hardness of input samples. |
| Outcome: | The proposed model outperforms or complements existing per-sample adaptive inference methods in terms of accuracy vs. FLOPs and can be applied to compressed and efficient transformer encoders to further improve their efficiency. |
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Transformer-based language models such as BERT, DeBERTa, and GPT3 have become the go-to tool in NLP. |
| Approach: | They propose an Early-Exit fine-tuning method that assigns each classifier its own set of unique model weights, not updated by other classifiers. |
| Outcome: | The proposed method outperforms Early-Exit and Multi-Model at fast speeds while maintaining comparable scores to Early- Exit at slow speeds. |
TACLR: A Scalable and Efficient Retrieval-based Method for Industrial Product Attribute Value Identification (2025.acl-long)
Copied to clipboard
Yindu Su, Huike Zou, Lin Sun, Ting Zhang, Haiyang Yang, Chen Li Yu, David Lo, Qingheng Zhang, Shuguang Han, Jufeng Chen
| Challenge: | Existing methods for product attribute value identification face critical challenges . seller-provided attribute values are often incomplete or inaccurate . |
| Approach: | They propose a retrieval-based method that uses taxonomy-aware contrastive learning . they use product profiles and candidate values to encode and retrieve attributes based on similarity . |
| Outcome: | The proposed method is based on a taxonomy-aware, hard negative sampling and adaptive inference with dynamic thresholds. |